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ABSTRACT 

The purpose of the study was to investigate why changing from multiple choice to structured assessment has an 
effect on students performance. The study used interviews, document analysis and observations to collect data. Purposive 
sampling was used to select study participants and all those selected were perceived to be information rich in relation to the 
problem under investigation Study findings point to the syllabus as a factor promoting the use of structured questions for 
assessment. The study also revealed that students do not perform well in structured questions due to three main factors 
being: teacher pedagogy skills, instructional and assessment language and teacher training on Agriculture and availability 
of learning materials. Revision of the syllabus and in-service training are recommended as possible remedies to the 
problem. 
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INTRODUCTION 

Primary education is the most important stage in the educational journey hence the Government of Botswana 
strives to make this level of education accessible to all through universal access to primary education, which runs for 
seven years. The country's Net Enrolment Ratio at primary schools stands at 95 % for children aged seven to thirteen years 
(State of the Nation address, 2010). Agriculture is an upper primary subject and is taught from Standard five to seven. 
Other subjects such as Mathematics, English, Science, Social Studies and Religious and Moral Education are taught from 
Standard one to seven. 

The pre-requisite for progression from Primary School to Junior Secondary School is Primary School Leaving 
Examination (PSLE) and Agriculture is one of the subjects that is examined. Initially, Agriculture was not examined at 
PSLE but from 2005, the Ministry of Education and Skills Development (MOESD) adopted a strategy for the Agriculture 
syllabus to be examined. Like other subjects, it was examined through objective testing (multiple choice). However, from 
2009, the assessment for Agriculture was changed from objective (multiple choice) to subjective testing 
(Structured questions). This change of assessment resulted in a drastic decline in performance. In 2008, the average 
performance was 71%, second only to Setswana at 78.3%. However, performance dropped drastically to 40.2%, 39.4% and 
23.2% for 2009, 2010 and 201 1 respectively (Botswana Examination Council, 201 1). 

Teachers, parents, students and the nation at large are concerned about this drastic decline in performance. 
What raises more concern is the fact that the decline was realised after the assessment structure was changed. 
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This study therefore, is an attempt to investigate why the assessment structure was change and why the change lowers 
performance. 

Several studies have been done to attempt an explanation to why students perform better in multiple choice as 
opposed to structured questions. Oyebola, Adewoye, Iyaniwura, Alada, Fasanmade and Raji (2000), conducted a 
comparative study of student's performance in preclinical physiology assessed by multiple-choice questions (MCQs) and 
structured questions (SQs). The results showed that students performed better in all MCQs tests than in SQs. 
Limited knowledge on the subject matter and inability to present and organise information logically on the part of the 
students were singled out as factors affecting performance in SQs. 

Most students prefer multiple choice to structured questions. The preference is borne from the belief that 
structured questions are difficult and multiple-choice questions are easier as one can still perform well in them under 
minimum preparation (Rocklin, 1992; Parmenter, 2009; Bridgeman, 1992). The attitude of students towards a particular 
assessment structure also affects performance (Coniam, 1999) and studies that investigated the correlation between scores 
from MCQs and SQs revealed a mixture of results ranging from no correlation to a strong and weak correlation 
(Walstad and Becker, 1994; Hancock, 1994). 

Chan & Kennedy (2002) did another comparative study comparing the easiness of multiple-choice exams to their 
structured equivalents. The main purpose of the study was to determine whether the average score on MCQs is 
significantly different from the average score on the equivalent SQs. The results of the t-test analysis revealed a higher 
average score in MCQs that in SQs equivalent. The study attributed the high mean score in MCQs to the nature of MCQs, 
which have alternative that guides students to deduce the correct answer. 

The findings of most of the studies reviewed point to the fact that average performance is likely to be high when 
learners are examined using MCQs as compared to SQs as is evidenced with PSLE Agriculture results in Botswana. 

METHODOLOGY 

The researchers used a qualitative case study approach. With the qualitative approach, the researcher has direct 
contact and gets close to the participants, situations and phenomenon under study (Johnson & Christensen, 2000). 
The approach is elaborate hence enables the researcher to collect detailed and in-depth information about the problem 
under study. Study participants were purposively drawn from a population of assessment experts, psychologists, teachers, 
students and lecturers. In all, forty one (41) people were interviewed consisting of four (4) Assessment specialists, three (3) 
Psychologists, twenty (20) Teachers, ten (10) Form One students and four (4) Lecturers. 

The sampled Form one students had just completed their PLSE Agriculture Examinations the previous year. 
Assessment specialists, teachers and lecturers provided in depth knowledge on assessment and pedagogy issues. Collection 
of data was done through one to one interviews, documents analysis (syllabus) and observations. In preparation for 
analysis, the data was coded and reduced into meaningful themes and patterns extracted to come up with meaningful 
conclusions. 

RESULTS AND DISCUSSIONS 

Study findings point to the syllabus as a factor dictating the choice of structured questions as a form of summative 
assessment and from the interviews; many factors were associated with students inability to perform well with structured 
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questions. The factors are fragmented into three related themes; Teacher pedagogy skills for aiding critical thinking, 
Instructional and assessment Language, and Agriculture Teacher training and availability of learning materials. 

The Upper Primary Agriculture Syllabus 

The analysis of the Agriculture syllabus revealed a high concentration of high order objectives. Out of the one 
hundred and nineteen (119) specific objectives in the syllabus, ninety-eight (98) are anchored with high order action verbs 
and only twenty-one (21) are anchored with low order verbs. This presents a highly ambitious and complex first experience 
syllabus. The syllabus does not take into consideration the context and reality in Primary schools. Analysing the syllabus, 
one might assume that it is a continuation of the lower primary syllabus, which could have covered low order specific 
objectives as baseline information, yet the upper Primary Agriculture is the first syllabus that exposes learners to 
Agriculture. What the syllabus expects in terms of specific learning outcomes makes it almost impossible to assess it using 
the multiple-choice structure. Eighty two percent (82%) of the specific objectives require pupils to demonstrate their 
comprehension, application and evaluation skills and those are best assessed through structured questions. There are few 
(18%) low order and knowledge focused specific objectives contained in the syllabus and this might explain why multiple 
choice was only used for the first years of PSLE assessment. 

Though teachers should prepare students to answer questions across all levels of cognitive thinking, most teachers 
at Primary level prepare students for low order questions that require simple recall. This can be attributed to the fact that all 
subjects except Agriculture are assessed through multiple choice, which is assumed to be relatively easier. Majority of 
questions asked for school based formative assessments are also multiple choice. This is what one of the interviewed 
teachers had to say; "Generally at Primary schools, we do not prepare students for questions that require complex mental 
processes and even the type of tests that we give are mainly multiple choice questions testing knowledge. The short answer 
questions we normally give are guided by low order action verbs like list, state, give. This raises a legitimate expectation 
on the part of the learners that the final examinations will also require such actions which is not normally the case with 
PSLE Agriculture" 

Structured questions requires critical thinking on the part of the learner whereas multiple choice do not require 
much of that (Ku, 2009). Most of the participants interviewed attributed high multiple-choice performance to guess work 
and simple recall. The attribution is supported by Mujeeb et al (2010), who urge that students with strong factual recall 
abilities score higher in multiple choice 

Teacher Pedagogy Skills for Aiding Critical Thinking 

Most of the teachers and psychologists interviewed decried the lack of a robust teacher training programme on 
early childhood education as a factor contributing to failure of students especially when required to apply high order 
thinking like in the case of PSLE structured Agriculture examination. Children have the ability to develop critical thinking 
skills but they need to be scaffolded by teachers. Hacker (1998) urges that though experience is limited at the primary 
school age, the process for cognitive regulation is evident and could be further enhanced by specific training and 
awareness. Teachers at lower class level are therefore expected to be fully equipped with pedagogy skills that can help 
learners to develop creative and calculative thinking. According to Mwamwenda (1996), a teacher has to understand the 
way the child communicates and learns across all stages of development. Knowing this will enable the teacher to use 
appropriate pedagogy approaches to maximise learning. 
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The Instructional and Assessment Language Factor 

The language of instruction and assessment for Agriculture is English yet majority of students do not understand 
it. Most of the teachers interviewed alluded to the fact that most Primary school pupils are poor at speaking, reading and 
writing English. A good understanding of the instructional language enables learners to extract relationships between 
letters and sound, leading to learners comprehending ideas, following arguments and detecting implications 
(Gentry, 2000). Without mastering English, it becomes very difficult for learners to answer structured questions, as those 
require them to construct sentences in English compared to multiple choice, which mainly requires them to read. 
According to the Revised National Policy on Education, English should be used as a medium of instruction from Standard 
two but it is not the case in school as teachers have a negative attitude towards English and continue to teach in the local 
language (Setswana) beyond Standard two. This negative attitude towards English by teachers is modelled to students 
hence are likely not to do well in English based examinations especially the structured ones. 

The problem of English language is also compounded by the fact that the majority of the learners do not go 
through pre-school education, they go straight from home to Standard one at the age of seven. It is at pre-school ages that 
foundations of instructional language can be better developed. Once students master the foundations of instructional 
language, it becomes easier for them to read, understanding and apply what they have learnt in a complex testing situation 
like structured questions. Ehri (1995) further argues that learning to read at an early elementary grade is one of the most 
important of all the development tasks. 

Teacher Training on Agriculture and Availability of Learning Materials 

Most of the participants interviewed, especially teachers and lecturers attributed the poor performance to lack of 
specialist Agriculture teachers in Primary schools. Agriculture is a relatively new subject at Primary school hence very few 
Primary school teachers are specialist in the subject. Teacher Training Colleges in the country are offering a generic 
Diploma in Primary Education with limited Agriculture content. Agriculture as a practical subject require teachers to apply 
pedagogy skills particular to it. Teaching Agriculture also requires a creation of a real life context by engaging learners in 
practicals but the teaching of Agriculture at Primary schools is wholly theoretical. The unique pedagogy and practical 
approach to the subject therefore require teachers who are well trained in the subject. Teachers interviewed expressed lack 
of confidence to deliver the syllabus, as there is also no in-service training on the subject. 

The practical aspects of the syllabus also requires learning requisites like a functioning garden and a school farm 
where learners can be able to apply their cognitive and psychomotor skills to real farming situations. All the schools 
observed had no gardens, tools or other farm structures where students could carry out practicals as expected by the 
syllabus. 

Students interviewed complained about shortage of textbooks and lack of other learning materials for Agriculture 
like a working school garden and farm. They also complained that some of the questions in the PSLE examination assume 
that they have fully functional farms with all the necessary requisites. To contextualise the students argument, below is an 
extract from the 2010 PSLE examination. 
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The diagram below shows an animal house. 




Source: BEC 2010 
Figure 1: An Extract Question from a Past PSLE Paper 

• Name the feature labelled H 

• State two reasons why feature H is important 

Despite the ambiguity and multiple effect of the question, it is a high order question that requires students to know 
the type of farmhouse, the features that make it as well as their functions. For a student to answer this question correctly, 
they must have seen and interacted with such a structure or at least saw it in the textbook but most schools do not have 
operational farm let alone enough textbooks. 

The lack of adequate teacher training on the subject and learning materials has resulted in teachers developing a 
negative attitude towards the subject and that has been transferred to students. 

CONCLUSIONS 

From the findings of the study, it is clear that teacher training on differentiated pedagogy and Agriculture are 
essential for teachers to effect positive observable change in the behaviour of leaners. The syllabus should also be 
pragmatic enough to allow for differentiated learning and the use of multiple assessment approaches. The use of English as 
an instructional and assessment language also contributes to low performance in structured examinations, as learners are 
not able to express themselves clearly in written English. 

The language and pedagogy factors are not unique to Agriculture. There is a high likehood that other subjects will 
also perform poorly if assessment through structured questions therefore comparing and ranking performance from 
multiple choice with that of a non-equivalent structured examination might not be the best approach for appreciating 
performance across subjects. 

RECOMMENDATIONS 

Based on the findings of the study, the following recommendations are made: 

• In-service workshops should be organized for teachers to capacitate them on Agriculture 

• PSLE assessment should combine multiple choice and structured questions for all subjects 

• Primary schools must be resource with Agriculture learning resources like gardens, tools and textbooks 

• The Primary Agriculture syllabus should be revised to make it cognitively balanced. 
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